Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use proper HTTP client for fetching credentials #2041

Merged
merged 2 commits into from
Dec 27, 2024

Conversation

ramondeklein
Copy link
Contributor

This PR ensures that the HTTP client that is set on the MinioClient will also be used when the client needs to fetch credentials via an HTTP request.

  • Instead of passing the HTTP client through all layers, I have choosen to use a CredContext structure that abstracts the HTTP client and may also be used to add more context (when needed).
  • The STSCertificateIdentity method passes certificates via the TLSClientConfig, so in this case the HTTP transport is cloned and it sets the certificate explicitly. The original implementation did set additional http.Transport fields, but this has been removed.
  • The credentials.(*Credentials).Get() function is deprecated and Provider.GetWithCredContext() should be used instead. It's still there to provide backward compatibility.
  • When this PR is merged, then we should also update madmin to also pass the custom HTTP client.

Fixes #2040.

@ramondeklein ramondeklein self-assigned this Dec 26, 2024
@ramondeklein ramondeklein marked this pull request as draft December 26, 2024 11:49
@ramondeklein ramondeklein marked this pull request as ready for review December 26, 2024 11:50
@ramondeklein ramondeklein force-pushed the http-client-fix branch 3 times, most recently from f1d5c43 to ed02e8d Compare December 26, 2024 12:12
Comment on lines -603 to +605
var retryable bool // Indicates if request can be retried.
var bodySeeker io.Seeker // Extracted seeker from io.Reader.
var reqRetry = c.maxRetries // Indicates how many times we can retry the request
var retryable bool // Indicates if request can be retried.
var bodySeeker io.Seeker // Extracted seeker from io.Reader.
reqRetry := c.maxRetries // Indicates how many times we can retry the request
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was changed by my IDE to follow coding guidelines.

pkg/credentials/sts_tls_identity.go Outdated Show resolved Hide resolved
Comment on lines +123 to +124
trCopy := tr.Clone()
trCopy.TLSClientConfig.Certificates = []tls.Certificate{i.Certificate}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was the best alternative that I could come up with. We don't want to change the TLSClientConfig of the original HTTP transport, because that may have security and parallelization issues.

@ramondeklein ramondeklein force-pushed the http-client-fix branch 2 times, most recently from c1c6c07 to f9f409f Compare December 26, 2024 19:17
Comment on lines +40 to +42
if i.Client == nil {
i.Client = &http.Client{}
}
Copy link
Contributor Author

@ramondeklein ramondeklein Dec 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously, the i.Client was set when NewSTSCertificateIdentity created the STSCertificateIdentity structure. So we need to check for nil here, because the default HTTP client is now nil.

Comment on lines +147 to +149
// Clone the HTTP client (patch the HTTP transport)
clientCopy := *client
clientCopy.Transport = trCopy
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although not the most elegant solution, I guess this is the safest method to ensure we get a copy of the HTTP client.

@ramondeklein
Copy link
Contributor Author

@harshavardhana I think I addressed your comments. If you prefer, I could add some unit-tests to ensure that the custom transport is being invoked. Not sure if that's needed.

@harshavardhana harshavardhana merged commit 4a691e1 into minio:master Dec 27, 2024
5 checks passed
@ramondeklein ramondeklein deleted the http-client-fix branch December 27, 2024 08:57
@klauspost
Copy link
Contributor

@ramondeklein I though this code was on the "Unesco World Heritage List" - nobody has been touching it for years :)

LGTM :)

harshavardhana added a commit that referenced this pull request Dec 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Custom HTTP transport is not used for STS
3 participants